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1 .  Abstract 

During  this  contract  period  we  have:  (a)  continued  investigation  of 
events  and  actions  by  means  of  representation  schemes  called  "event  shape 
diagrams";  (b)  written  a  parsing  program  which  selects  appropriate  word  and 
sentence  meanings  by  a  parallel  process  known  as  activation  and  inhibition; 
(c)  begun  investigation  of  the  point  of  a  story  or  event  by  modeling  the 
motivations  and  emotional  behaviors  of  story  characters;  (d)  started  work  on 
combining  and  translating  two  machine-readable  dictionaries  into  a  lexicon  and 
knowledge  base  which  will  form  an  integral  part  of  our  natural  language  under¬ 
standing  programs;  (e)  made  substantial  progress  toward  a  general  model  for 
the  representation  of  cognitive  relations  by  comparing  English  scene  and  event 
descriptions  with  similar  descriptions  in  other  languages;  (f)  constructed  a 
general  model  for  the  representation  of  tense  and  aspect  of  verbs;  (g)  made 
progress  toward  the  design  of  an  integrated  robotics  system  which  accepts 
English  requests,  and  uses  visual  and  tactile  inputs  in  making  decisions  and 
learning  new  tasks. 

2 .  Introduction 


During  the  past  reporting  period,  we  have  concentrated  on  understanding 
language  that  requires  the  representation  of  time,  space,  physical  mechanisms, 
causal  relationships,  and  the  motivations  of  event  participants.  Major  new 
ideas  and  research  methods  during  the  past  year  include  the  following: 


a.  The  use  of  "event  shape  diagrams"  to  represent  verb  and  adverb  meanings. 
Event  shape  diagrams  are  based  on  the  idea  that  verb  meanings  can  be 
decomposed  into  a  set  of  predicates  whose  values  vary  together  through 
time.  Thus,  the  verb  hit  can  be  decomposed  into  the  following  predi¬ 
cates:  trajectory,  position,  velocity,  contact,  force,  etc.  These  predi¬ 
cates,  along  with  adverbs  (quickly,  hard,  softly,  etc.)  and  knowledge  of 
object  properties,  allow  programs  to  make  inferences  aboat  object 
behavior  such  as  breaking .  bending .  shattering .  deflecting,  etc.1 

b.  We  have  programmed  a  new  type  of  model  of  natural  language  processing 
that  selects  appropriate  meanings  for  words  and  sentences  by  a  process  of 
activation  and  inhibition  between  different  word  senses  and  the  context 
in  which  words  are  used. 


*A  detailed  account  of  event  shape  diagrams  was  reported  in  our  last  pro¬ 
gress  report,  T-114,  Coordinated  Science  Laboratory,  University  of  Illinois, 
May  1982. 
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c.  We  have  begun  investigation  of  the  point  or  moral  of  a  story  or  event 
description.  This  work  involves  modeling  the  motivation,  reasoning,  and 
emotional  behavior  of  both  the  participants  and  the  narrator  of  the  story 
or  event,  and  understanding  the  metaphorical  usage  of  words.  Some  prel¬ 
iminary  programming  work  has  been  done  on  metaphor  understanding  based  on 
event  shape  diagrams. 


d.  We  are  designing  mechanisms  for  automatically  combining  and  translating 
two  machine-readable  dictionaries  into  a  lexicon  and  knowledge  base  that 
will  form  an  integral  part  of  our  natural  language  understanding  pro¬ 
grams.  Different  word  senses  will  be  represented  by  a  set  of  special 
predicates  called  primitives,  by  event  shape  diagrams,  and  other  schemas 
we  are  now  investigating. 


e.  We  are  constructing  a  general  model  for  the  representation  of  role-based 
cognitive  relations.  Evidence  from  four  languages  (English,  Spanish, 
Basque,  and  Mandarin)  indicates  that  relations  between  actors  and  objects 
can  be  expressed  syntactically  (for  example,  by  the  use  of  word  order), 
morphologically  (for  example,  by  means  of  word  endings)  and  semantically 
(for  example,  via  semantic  roles  such  as  actor ,  goal,  location,  t ine , 
etc.).  These  primary  distinctions  are  intended  to  serve  as  the  basis  for 
a  representation  language  which  enables  us  to  map  natural  language 
expressions  onto  cognitive  representations,  and  vice  versa. 

f.  Tense  and  aspect  of  verbs  have  been  studied  for  several  languages,  and  a 
general  representation  model  constructed.  Tense  and  aspect  refer  to  the 
meaning  changes  induced  by  adding  auxiliary  constructions  to  verbs.  Our 
model  aims  to  systematically  assign  appropriate  meaning  structures  to 
phrases  such  as:  would  have  hit,  had  hit,  attempted  to  hit.  almost  hit, 
and  allowed  to  hit.  These  models  are  now  being  programmed  into  an  event 
meaning  representation  system. 

g.  We  are  designing  a  robust  robotics  system  which  will  use  natural  language 
input  and  feedback  from  human  language  users  to  make  decisions  and  per¬ 
form  goal-based  tasks  The  system  is  based  on  a  high-level  programming 
language  which  represents  a  task  as  a  series  of  goals  and  subgoals. 
Knowledge  about  goals  enables  the  system  to  request  information  from 
other  systems  (for  example,  visual  or  tactile)  when  completion  of  a 
subgoal  required  to  achieve  a  main  goal  is  blocked.  The  system  will  also 
rely  on  information  from  its  subsystems  in  the  process  of  learning  about 
new  tasks. 
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3.  Parallel  Ne twork  Natural  Language  Processing 

3.1.  Goals  of  this  Research 

Natural  language  processing  requires  the  cooperation  of  many  kinds  of 
knowledge,  both  language-specific  knowledge  and  "real-world"  knowledge.  Much 
of  the  current  research  in  NLP,  in  this  research  group  and  elsewhere,  is 

concerned  with  the  problems  in  the  representation  of  knowledge.  Our  research 
on  parallel  network  natural  language  processing,  on  the  other  hand,  addresses 
itself  to  the  problems  in  the  cooperative  application  (or,  interactive 

integration)  of  knowledge. 

Specifically,  we  have  been  looking  at  highly  parallel  and  analogue 
techniques  for  the  cooperative  application  of  knowledge  expressed  in  standard 
structures  (e.g.  grammars,  case  structures,  scripts,  etc.)  to  language 

processing.  (Evidence  for  parallel  integrated  processing  has  been  reported 
[Marslen-Wil son  and  Tyler,  1980],  and  AI  researchers  have  started  talking  of 
integrated  processing  [Schank  and  Birnbaun,  1980],  but  the  models  put  forth 
have  been  serial  and  top-down.) 

In  a  concurrent  effort,  we  are  looking  towards  parallel  computer 

architectures  to  support  these  techniques. 

3.2.  Methods 

Our  method  is  an  experimental,  and  highly  computational  one.  Rather  than 
taking  a  tightly  circumscribed  subproblen  of  language  processing,  collecting 
examples  of  how  humans  deal  with  the  subproblem,  and  writing  a  program  which 
approximates  it,  we  are  looking  at  the  kinds  of  decisions  needed  in  a  host  of 
subproblems,  assuming  that  these  decisions  need  to  interact,  and  we  are 
hypothesizing  mechanisa\s  which  can  account  for  both  the  decisions  and  their 

2 

interactions.  Thus  we  do  not  suffer  from  the  "big  switch  "  problem,  nor  the 
3  ... 

"chicken  and  egg  "  problem,  but  still  have  problems,  mainly  in  the  lack  of 
expressive  power  of  our  computers.  With  this  in  mind,  we  are  scaffolding  our 
way  to  a  natural  language  understanding  system  through  a  series  of  programs, 
each  building  on  and  refining  its  predecessor. 

Currently,  our  model  fuses  ideas  from  semantic  network  representation 
schemes,  including  structured  inheritance  [Brachman,  1978],  marker-passing 
algorithms  [Quillian,  1968],  and  virtual  copies  [Fahiman,  1979],  with  concepts 


2 

If  A.  writes  a  program  for  syntactic  analysis,  B.  writes  a  program  for 
pronoun  resolution,  C.  writes  the  noun  phrase  handler,  D.  writes  a  script 
selector,  etc.,  how  do  they  fit  together? 

3 

Researchers  still  argue  whether  the  "syntax  box"  feeds  the  "semantics  box" 
or  vice  versa. 

4 

The  current  serial  computer  architectures  and  languages  are  computational¬ 
ly  powerful  enough  (theoretically)  but  not  expressively  powerful  enough  to  en¬ 
able  us  to  easily  write  the  programs  we  envision.  Consider  writing  a  PL/1  com¬ 
piler  on  a  Turing  machine) 
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froa  cooperative  computation,  mainly,  spreading  activation  and  lateral 
inhibition  [McClelland  and  Rumelhart,  1981;  Feldman  and  Ballard,  1982] 

More  specifically,  we  are  designing  a  virtual  machine  which  will 
uniformly  handle  several  aspects  of  language  processing  which  have, 
historically,  used  different  "boxes",  including: 

(1)  assignment  of  syntactic  structure; 

(2)  selection  of  lexical  categories; 

(3)  selection  of  word  senses; 

(4)  filling  of  case  roles; 

and 

(5)  use  of  contextual  constraints. 

The  current  machine  model  includes  several  concurrent  processes  that 
operate  on  a  weighted,  labeled  network:  Input  stimuli  (i.e  words)  cause 

breadth  first  instantiation  of  certain  nodes  and  their  active  neighboring 
concepts  into  a  "short  term  memory,"  where  patterns  (already  in  this  memory) 
"fire",  and  perform  purposeful  connection,  of  the  newly  created  nodes  to 
existing  structure.  The  dual  processes  of  spreading  activation  and  lateral 
inhibition  work  together  to  select  well-connected  and  consistent  structures 
over  disconnected  fragments,  and  those  nodes  that  become  inhibited  past  a 
certain  threshold  are  subject  to  termination,  whereby  they  are  disconnected 
from  the  network. 

This  model  is  similar  to  research  in  progress  at  the  University  of 
Massachusetts,  [Gigley,  1982],  at  Carneg ie-Mel Ion  [Thibadeau,  Just,  and 
Carpenter,  1982],  at  the  University  of  Rochester  [Cottrell,  1982],  and  at  U.C. 
San  Diego,  but  is  much  more  computationally  oriented  than  any  of  them. 

3.3.  Progress 

Progress  has  been  made  over  the  past  year  in  several  areas,  and  each  will 
be  discussed  separately. 

3.3.1.  Progress  in  Application 

Working  on  small  examples,  we  have  shown  the  applicability  of  this  style 
of  processing  to  the  interactions  between  various  kinds  of  knowledge, 
including  lexical,  syntactic  and  semantic.  For  example,  consider  the 
(syntactically)  ambiguous  sentence: 

John  ate  up  the  street. 

We  manually  set  up  a  network  for  this  sentence  based  on  the  output  of  a  chart 
parser  [Kay,  1973],  Using  a  proportional  activation/ inhibition  process 
[McClelland  and  Rumelhart,  1981]  the  network  was  confused  at  first,  but 
stabilized  on  the  locational  reading  of  "up".  Next,  the  weights  in  the  network 
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were  adjusted  to  represent  a  lexical  preference  for  interpreting  "up"  as  an 
particle,  and  the  network  quickly  selected  all  nodes  participating  in  the 
conical  reading  of  the  sentence,  the  one  in  which  John  ate  concrete.  Finally, 
some  semantics  were  added,  filling  in  the  LOCATION  case  of  "ate”  with  "up  the 
street"  and  the  normal  sentence  reading  was  again  selected.  These  results  were 
reported  at  the  Cognitive  Science  Society  *982  Conference  in  August  [Pollack 
and  Waltz,  1982]  . 

One  of  the  original  uses  devised  for  spreading  activation  systems  was 
memory  priming  [Collins  and  Quillian,  1972]  and  context  [Ortony,  1976].  We 
have  experimented  with  this  type  of  contextual  pressure  in  the  sentence: 

John  shot  some  bucks 

In  a  null  context,  the  constructed  network  was  unable  to  make  a  decision 
between  the  "fired  at"  and  "wasted"  readings  of  "shot"  and  the  corresponding 
"deer"  versus  "money"  reading  of  "bucks",  but  when  the  network  was  slightly 
perturbed  in  favor  of  either,  (as  if  there  were  a  proceeding  context  "While 
hunting,"  or  "While  gambling”)  the  confusion  rapidly  cleared  up.  These  results 
will  be  reported  in  in  detail  in  a  forthcoming  paper. 

3_.3,.2.  Progress  in  Computational  Models 

Four  versions  of  the  program  have  been  written,  incorporating  more  and 
more  of  the  proposed  model.  The  first  version  did  not  have  labeled  links,  and 
was  just  an  experimental  vehicle  for  understanding  the  nature  of  activation 
and  inhibition.  The  second  version  incorporated  a  primitive  semantic  network 
superimposed  on  the  weighted  network,  but  had  no  dynamic  construction 
capability;  the  networks  had  to  be  built  by  hand.  The  third  version  included 
some  of  the  ideas  on  breadth-first  instantiation  and  connection,  but  the 
pattern  system  was  not  robust  enough.  The  fourth  version  is  undergoing 
refinement  (read  "debugging")  currently  and  has  incorporated  several 
efficiency  and  expressiveness  improvements,  including  the  removal  of 
floating-point  operations  in  favor  of  scaled  integers,  and  a  better  language 
for  network  initialization. 

3  .3[.3  .  Progress  in  Architecture 

Probably  the  most  important  question  in  Computer  Science  today  is  how  to 
effectively  build  and  program  highly  parallel  architectures  to  take  ad-antage 
of  the  economies  of  large  scale  integrated  circuitry^. 

With  this  in  mind,  we  have  designed  part  of  an  architecture  for  the 
parallel  simulation  of  activation/inhibition  networks.  It  is  organized  as  a 
linear  array  of  cells  in  NMOS.  Each  cell  in  the  hardware  corresponds  to  a  node 
in  a  network,  and  has  a  register  holding  its  activation  value  and  a  self- 
sorting  array  holding  its  links.  In  a  fixed  and  finite  amount  of  time,  all  the 
cells  can  send  contributions  to  a  programmed  set  of  other  cells,  receive  all 


[Backus,  19781  points  out  the  need  for  alternative  models  of  computation 
to  free  the  mind  of  the  shackles  of  seriality;  [Sutherland  and  Mead,  1977] 
discuss  the  constraints  on  VLSI  architecture  design. 
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the  contributions  sent  to  them,  and  update  their  activation  registers.  The 
crucial  element  in  the  achievement  of  this  performance  is  the  recognition  that 
messages  heading  towards  the  same  location  may  be  added  together  because  they 
are  numeric.  The  architecture  takes  advantage  of  this  knowledge  by  adding 
messages  whenever  possible,  rather  than  after  they  reach  their  destination. 
More  details  of  this  architecture  are  given  in  Working  Paper  31  [Pollack, 
1982]  . 

3.. 3. 4.  Progress  in  Theory 

Besides  continuing  work  on  refinement  of  the  model  and  of  the  program, 
and  on  their  application  to  more  difficult  problems  in  NLP  such  as  pronoun 
resolution  and  nominal  compounds,  we  are  investigating  the  formal  mathematical 
behavior  of  such  networks.  Activation  and  inhibition  approximately  solve  the 
following  graph-theoretic  problem: 

Given  a  graph,  G  =  {V,E}  where  E  is  the  union  of  two  disjoint  sets  B 
(black  edges)  and  R  (red  edges),  find  a  maximal  subset  of  V,  V',  such 
that  the  graph  induced  by  V*  is  black-connected  with  no  red  edges. 


Consequently,  we  are  working  to  understand  the  computational  complexity  of 
this  problem:  If  it  is  NP-complete  (or  NP-hard) ,  then  activation  and 
inhibition  may  provide  a  good  parallel  approximation  algorithm  for  these 
difficult  problems;  if  it  is  P(olynoaial) ,  then  there  may  be  a  very  efficient 
way  to  program  a  network  simulation  on  current  generation  computers. 
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4.  The  Role  of 
4.1.  Goals  and 


Affect  in  Narrative  Comprehens ion 
Impl icat ions  of  the  Research 


The  goal  of  the  present  study  is  to  develop  a  structural  model  of  emotion 
which  will  be  suitable  for  computer  implementation.  Following  Lehnert's 
definition  of  an  affect  unit**  the  first  part  of  the  study  will  attempt  to 
identify  affect  units  which  commonly  occur  in  short  stories,  such  as  fables  or 
folktales.  Once  a  set  of  affect  units  has  been  identified,  children  and 
adults  will  be  experimentally  tested  on  their  ability  to  recognize  affect 
units,  and  to  make  inferences  on  the  basis  of  affective  information.  The 
final  stage  of  the  project  will  consist  of  the  construction  of  a  computer 
model  for  story  understanding  which  is  capable  of  a 
Ideally,  the  implementation  will  reflect  aspe 
consistent  with  the  experimental  findings.  The  imp 
research  are  two-fold:  (1)  to  construct  a  highly 
language  processing,  and  (2)  to  advance  cognitive 

knowledge  representation,  language  understanding,  inference,  and  learning. 
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4.2.  Previous  Research 


While  language  understanding  systems  in  artificial  intelligence  have 
focused  on  knowledge  structures  such  as  frames  [Minsky,  1975],  scripts  [Schank 
and  Abelson,  1977]  and  goals  [Wilensky,  1978],  considerably  less  attention  has 
been  devoted  to  the  role  of  affect  in  natural  language  processing.  In 
psychology,  there  have  been  a  number  of  attempts  to  develop  a  structural 
theory  of  the  emotions  [Osgood,  1966;  Roseman,  1979,  in  preparation;  and 
Brewer,  in  press],  Roseman,  for  example,  indicates  that  emotional  reactions 
are  based  on  the  cognitive  dimensions  Availability,  Desirability,  Probability, 
Agency,  and  Legitimacy.  In  various  combinations,  these  dimensions  give  rise 
to  13  discrete  emotions:  Joy,  Relief,  Hope,  Liking,  Pride,  Distress,  Sorrow, 
Fear,  Frustration,  Disliking,  Anger,  Regret,  and  Guilt.  Anger,  for  example, 
may  result  from  the  perception  that  some  agent  (Agency  dimension)  is 
preventing  us  (Probability  dimension)  from  obtaining  some  goal  (Availability 
dimension)  that  we  want  (Desirability  dimension)  and  deserve  (Legitimacy 
dimension).  An  early  goal  is  to  compare  Roseman's  work  with  similar  taxonomies 
in  order  to  identify  redundant  categories  and/or  possible  omissions. 

In  artificial  intelligence,  interest  in  emotion  has  been  promoted  by 
research  on  memory  representation  and  story  summarization  [Lehnert,  1980; 
Dyer,  1982]  .  Lehnert  claims  that  narratives  can  be  represented  in  terms  of 
three  primary  affect  states:  states  associated  with  positive  events,  states 
associated  with  negative  events,  and  states  associated  with  internal  needs  or 
desires.  In  addition,  Lehnert  specifies  four  types  of  connections,  or  affect 
links,  which  describe  an  oriented  arc  between  any  two  states.  A  MOTIVATION 


Lehnert  uses  the  term  "affect  unit"  to  describe  affective  causal  relations 
among  events  in  a  narrative.  Hie  affective  unit  for  retaliation,  for  example, 
consists  of  the  following  causal  sequence:  A  given  event  results  in  X’s  suc¬ 
cess  and  Y's  failure.  Y's  failure  motivates  him  to  bring  about  an  event  which 
ensures  his  own  success  and  X's  failure.  I  experiences  happiness  as  a  result 
of  X's  misfortune. 
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link,  for  example,  connects  a  positive  or  negative  event  to  a  need  state. 
Other  affect  links  include  ACTUALIZATION  links.  TERMINATION  links,  and 
EQUIVALENCE  links.  According  to  Lehnert,  states  and  links  can  combine  to 
prodace  affect  units,  which  are  capable  of  representing  different  types  of 
plot  structares. 

While  Lehnert's  work  provides  an  appropriate  starting  point  for  the 
identification  of  emotional  primitives  and  complex  emotional  anits.  Lehnert's 
representation  system  requires  further  empirical  justification.  For  example, 
are  affect  units  reliably  recognized  by  readers?  How  are  readers  able  to 
make  inferences  on  the  basis  of  affective  information  which  is  not  explicitly 
stated  in  the  text?  How  does  affect  interact  with  other  types  of  knowledge 
structures,  such  as  beliefs  or  values?  These  are  some  of  the  questions  which 
we  intend  to  address  in  the  present  research,  hopes  to  address. 


Methqdolos 


The  present  study  will  be  carried  out  in  several  stages,  employing 
research  methods  from  experimental  psychology  and  artificial  intelligence. 
The  first  stage  (currently  underway)  is  to  identify  (1)  a  set  of  emotional 
primitives,  and  (2)  a  set  of  affect  units  which  commonly  occur  in  short 
stories,  such  as  fables  or  folktales.  Once  a  set  of  affect  units  has  been 
identified,  children  and  adults  will  be  tested  on  their  ability  to  recognize 
affect  units,  and  to  make  inferences  on  the  basis  of  implicit  and  explicit 
affective  information.  The  hypotheses  which  will  be  tested  are  as  follows: 


(1)  The  recognition  of  affect  units  is  necessary  for  the  comprehension  of 
short  stories,  such  as  fables  or  folktales. 


(2)  Stories  which  contain  well-formed^  affect  units  are  more  cohesive  and, 
thus,  easier  to  understand  than  stories  which  don't. 


(3)  There  is  a  developmental  progression  in  the  ability  to  make  affective 
inferences  on  the  basis  of  inexplicit  affective  information.  That  is, 
the  youngest  children  will  have  the  most  difficulty  interpreting  stories 
in  which  some  degree  of  empathy  or  emotional  perspective-taking  is 
required. 

(4)  There  is  a  developmental  progression  in  the  ability  to  generalize 
familiar  affect  units  to  new  situations.  That  is,  the  youngest  children 
will  have  the  most  difficulty  interpreting  stories  which  consist  of 
affect  units  which  must  be  learned  by  analogy  to  a  preceding  story. 

The  final  stage  of  the  project  will  include  the  construction  of  a  story 
understanding  system  which  will  be  capable  of  making  affective  inferences  on 

7 

Presently,  I  know  of  no  independent  criteria  to  support  the  notion  of 
well-formedness  for  affect  units.  Hopefully,  we  will  be  able  to  predict  ac¬ 
ceptable  patterns  of  co-occurrence  and  sequential  occurrence  of  emotional 
reactions  by  means  of  statistical  techniques  such  as  factor  analysis. 
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the  basis  of  (often)  incomplete  or  inexplicit  information.  Ideally,  the 
implementation  will  reflect  aspects  of  human  performance  consistent  with  the 
experimental  findings  (for  example,  what  kind  of  information  is  required  for 
affective  frame  selection  and  role  instantiation).  Finally,  constructing  a 
model  of  affect  will  require  an  understanding  of  the  interaction  between 
affective  knowledge  and  other  knowledge  sources,  such  as  beliefs  and  values. 
Since  such  topics  have  been  relatively  unexplored  in  artificial  intelligence, 
the  construction  of  a  fully  integrated  system  of  the  sort  proposed  here  is 
particularly  timely. 
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5 .  Automatic  Lexicon  Building 
5.1,.  Goals 

This  paper  describes  work  aimed  at  the  eventual  construction  of  an 

O 

automatic  lexicon-builder  for  natural  language  processors.  The  lexicon- 
builder  will  use  the  vast  amount  of  information  in  common  dictionaries, 
transducing  it  into  a  form  more  appropriate  for  natural  language  processing. 
A  major  intermediate  step  in  the  project  is  the  formulation  of  an  internal 
representation  sufficient  to  record  all  the  information  contained  in  common 
dictionaries.  The  availability  of  an  automatic  lexicon-builder  would  enable 
rapid  development  of  natural  language  processors  by  freeing  the  system 
developer  from  hand-coding  large  lexicons,  therby  enabling  him/her  to 
concentrate  on  larger  problems. 

5.2.  Background 

Natural  language  processors  store  a  great  deal  of  their  knowledge  in  word 
definitions  and  in  schemas  .  Assembling  this  knowledge  by  hand  is  becoming 
impractical  as  increasingly  larger  and  more  comprehensive  lexicons  become 
necessary.  Vast  lexico-semantic  knowledge  will  be  needed  for  industrial 
strength  natural  language  processors,  and  perhaps  more  pressingly,  for 
advanced  research  in  Artificial  Intelligence,  Linguistics,  Cognitive  Science, 
and  other  areas.  DeJong's  work  in  schema  learning  [DeJong,  1982]  addresses 
this  issue  on  the  level  of  "real-world  knowledge."  Automated  assembly  would 
certainly  speed  the  construction  of  larger  lexicons,  and  in  all  probability, 
yield  a  more  thorough  and  internally  uniform  product  than  hand  construction. 

Dictionaries  may  be  of  some  help  in  automating  lexicon  construction. 
They  contain  morphological,  syntactic,  semantic  and  usage  information  that 
must  be  included  in  a  lexicon.  We  should  be  able  to  take  advantage  of  the 
inherent  structure  and  regularity  of  dictionaries.  Amsler  [Amsler  and  White, 
1979]  has  already  begun  to  catalogue  some  of  that  structure.  Further  work  in 
this  area  is  vital. 

5.3 .  Knowledge  Representation  for  Verb  Meaning 

One  important  goal  of  A. I.  work  in  natural  language  is  the  formulation  of 
a  representation  system  adequate  to  encode  the  knowledge  humans  use  to 
communicate  about  their  world.  Many  formalisms  have  been  invented  for 
representing  both  highly  specialized  and  general,  common-sense  knowledge. 
Schank's  system  [Schank  and  Riesbeck,  1981;  Schank,  1975]  is  very  powerful, 
but  it  is  not  "fine-grained"  enough  to  capture  many  important  details  of 


g 

In  the  following  discussion,  I  use  the  word  dictionary  to  mean  the  printed 
(or  machine-readable)  books  published  by  such  companies  as  Random  House, 
Merriam-Webster ,  etc.  By  lexicon.  I  mean  the  component  of  a  natural  language 
processor  which  holds  the  program's  knowledge  about  words. 

9 

Frames,  scripts  and  schemas  belong  to  the  same  genus  of  data  structure. 
The  word  "schema"  is  used  here  to  refer  to  all  three,  since  schemas  are  the 
most  general  of  the  group. 
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meaning.  For  example,  in  Schank's  system  the  verbs  is  Figure  1  all  map  into 
the  same  conceptual  dependency  primitive,  INGEST.  The  similarities  are 
apparent,  yet  the  differences  in  meaning  evaporate  when  all  these  verbs  are 
reduced  to  their  common  element  of  "motion-resulting-in-containment . "  [Waltz, 
1982] 

In  addition,  there  are  many  verbs  whose  representation  within  the  system  is 
grossly  inadequate.  The  verbs  in  Figure  2  all  map  into  the  conceptual 
dependency  primitive  STATE-CHANGE.  Clearly  they  all  do  involve  some  change  of 
state,  but  the  exact  nature  of  the  change  is  not  specified.  A  more  powerful 
representation  is  certainly  required  to  record  and  reason  with  a  fuller  range 
of  the  nuances  available  to  natural  language  speakers. 

Although  dictionaries  define  words  in  terms  of  other  words,  and  not  in 
terms  of  conceptual  primitives,  they  still  embody  a  representation  system.  It 
may  be  that  the  words  used  in  definitions  are  just  synonyms  for  unnamed 
primitives.  If  so,  this  will  be  revealed  by  careful  study.  We  may  also  hope 
to  extract  the  relations  between  words  that  dictionaries  employ  in  definitions 
(e.g.  IS-A;  IS-PART;  various  case-argument  roles  such  as  INSTRUMENT,  AGENT, 
OBJECT;  causal  relations;  topological  relations;  spatial  relations; 
evolutionary  relations;  and  others).  Because  they  define  words  using  more 
words,  it  is  inevitable  that  dictionaries  contain  cycles.  I  expect  that  these 
cycles  are  not  meaningless,  but  rather,  may  conceal  information  that  could 
lead  us  to  the  abovementioned  primitves  and  relations.  Dictionary  studies 
will  serve  to  justify  the  relations  that  are  currently  employed  in  Artificial 
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Intelligence  work,  and  will  likely  torn  up  some  new  ones.  Nearly  every 
concept  that  has  a  name  in  English  has  an  entry  in  a  good  dictionary.  This 
leads  as  to  expect  that  a  representation  system  derived  from  dictionaries  will 
be  closer  to  adequate  than  cnrrent  systems. 

5.4.  Limits  of  Dictionaries 

Clearly,  there  are  limits  to  what  we  can  learn  from  dictionaries.  There 
are  issues  they  do  not  address.  For  example,  the  American  Heritage  Dictionary 
[Morris,  1970]  defines  "needle"  as: 

1.  A  small,  slender  sewing  implement,  now  usually  of  polished  steel,  pointed 
at  one  end,  and  having  an  eye  at  the  other  through  which  a  length  of 
thread  is  passed  and  held. 

But  it  says  nothing  about: 


a)  how  the  needle  is  used  in  sewing  (i.e.  that  it  is  forced  through  the 
materials  to  be  joined,  pointed  end  first,  dragging  the  thread  through 
the  hole  it  has  made,  causing  the  thread  to  bind  the  materials  together; 

b)  how  small  is  small; 

c)  what  they  used  to  be  made  of; 

d)  how  the  thread  is  held  in  the  eye; 

e)  what  is  an  "eye?" 

"Sew"  is  defined  as: 

1.  To  make,  repair,  or  fasten  with  a  needle  and  thread. 

2.  To  furnish  with  stitches  for  the  purpose  of  closing,  fastening, 

attaching,  or  the  like.  Often  used  with  up:  sew  up  a  wound. 

We  still  have  no  information  about  how  or  why  sewing  works  (see  point  a, 

above).  For  "stitch"  we  have: 

1.  A  single  complete  movement  of  a  threaded  needle  in  sewing  or  surgical 

suturing. 

2.  A  single  loop  of  yarn  around  a  knitting  needle  or  similar  implement. 

3.  The  link,  loop,  or  knot  made  in  this  way. 


The  problem  is  that  our  dictionary  does  not  tell  us  about  the  process  of 
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sewing.  In  exploring  dictionaries  for  the  purposes  outlined  above,  it  is  a 
good  idea  to  keep  track  of  the  places  where  dictionaries  are  inadequate.  From 
there  we  can  proceed  to  find  other  sources  for  the  missing  information. 

5.. 5  .  Conclusions 

To  summarize,  my  research  is  currently  oriented  along  the  following 
1 ines . 

1.  Analyzing  dictionaries  to  uncover  their  structure  both  within  word 
entries,  and  on  the  macroscopic  level.  Progress  in  the  areas  below  will 
hinge  on  the  results  of  this  analysis. 

• 

2.  Formulating  a  representation  scheme.  This  includes  isolating  primitives, 
and  developing  a  logic  for  primitives  and  relations. 

3.  Devising  and  testing  methods  for  converting  a  dictionary  to  a  lexicon. 

4.  Determining  what  information  cannot  be  fou:\d  in  dictionaries. 

5.  Finding  other  sources  to  compensate  for  this  deficiency. 

In  conclusion,  I  believe  that  the  dictionary  is  a  large  untapped 
resource.  My  goals  are  to  use  dictionaries  to  facilitate  the  assembly  of 
large  lexicons;  to  improve  currently  used  representation  systems  for  language; 
and  to  extend  our  understanding  of  words  and  the  way  we  use  them.  The 
dictionary  is  not  a  cure-all,  but  it  is  a  useful  source  of  information  and  a 
revealing  object  for  study. 
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6 .  Modeling  Events 

6.1.  Goal 

The  goal  of  the  present  research  project  is  (1)  to  characterize  a  set  of 
role-relations  for  the  various  elements  involved  in  any  given  event  or  state- 
of-affairs  which  is  both  1 ingustical ly  relevant  and  cognitively  plausible,  (2) 
to  outline  a  system  for  representing  the  conceptualizations  associated  with 
natural  language  sentences  in  terms  of  those  role-relations  and  (3)  to  develop 
a  procedure  for  mapping  natural  language  expressions  onto  such  cognitive 
representations  and  vice  versa. 

By  developing  an  explicit  procedure  for  mapping  between  language  token 
and  conceptual  representation,  we  hope  not  only  to  shed  light  on  the 
theoretical  status  of  "primitive  cognitive  relations"  but  to  provide  a  sound 
basis  for  further  work  in  the  psychology  of  language  processing  as  well.  In 
addition,  such  a  theory  would  present  the  appropriate  descriptive  framework 
for  work  in  such  applied  areas  as  computerized  natural  language  processing, 
automated  translation,  and  so  on. 

6.2.  Structure  of  Research 

The  project  is  divided  into  four  parts. 

A.  The  classification  of  role  distinctions  and  the  systems  for  marking  these 
distinctions  on  the  basis  of  the  formal  properties  of  a  number  of  actual 
natural  languages. 

This  problem  has  been  approached  by: 

— preparing  a  survey  which  is  designed  to  elicit  the  role  distinctions  made 
and  the  strategies  employed  for  making  them  in  any  given  natural  language 
and  administering  the  survey  to  speakers  of  typologically  distinct 
languages  from  different  geographical  areas 

— analysing  the  data  gathered  with  an  eye  toward  establishing  a  hierarchy 
of  role  classes,  a  listing  of  the  distinctions  made  within  each  class, 
and  a  system  for  the  marking  of  these  distinctions. 

B.  The  classification  of  role  distinctions  on  the  basis  of  functional 
considerations  with  respect  to  cognitive  processing  (i.e.  reasoning, 
inferencing,  and  so  on) . 

This  problem  is  being  approached  by: 

— reviewing  proposals  for  role  distinctions  above  and  their  motivations  in 
works  dealing  with  event  analysis 

— reviewing  the  proposals  for  role  distinctions  and  their  motivations  in 
works  dealing  with  reasoning  and  inferencing  systems 

— developing  a  classification  of  roles  as  motivated  by  the  needs  of  these 
systems. 
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C.  The  development,  on  the  basis  of  comparing  and  contrasting  the  results  of 

the  previous  parts,  of  a  "neutral"  class  of  roles  which  would  be 
appropriate  for  the  characterization  of  a  conceptual  level  of 

representation  for  any  parser  operating  on  natural  language  input  or  of  an 
organizational  level  of  representation  for  any  natural  language  generator. 

D.  The  instantiation  of  the  results  of  the  analysis  in  the  form  of  a  natural 
language  interpretor-generator  which  would  be  tested  for  both  adequacy  and 
extensibility. 

6.3.  Summary  of  the  Progre s s  to  Date 

Work  was  begun  in  January  of  1982.  The  initial  concentration  has  been  two 
pronged  in  that  both  parts  A  and  B  have  been  addressed  simultaneously  although 
the  former  has  received  much  more  emphasis  up  to  now.  Of  course,  parts  C  and  D 
of  the  research  cannot  be  approached  until  parts  A  and  B  are  essentially 
completed. 

A.  A  summary  of  the  results  from  Part  A 

1.  The  preparation  and  the  administration  of  the  language  survey 

During  the  Spring  of  this  year  I  prepared  a  survey  consisting  of 
some  four  hundred  sentences  and  phrases  in  English  for  the  pu.-pose  of 
gathering  data  concerning  some  fifty  different  role  relations  which 
nminals  could  potentially  bear  with  respect  to  verbs  or  other  nominal  s. 
The  survey  was  developed  along  the  lines  of  the  Lingua  Descript  ive 
Series  Question-1. ire  [Comrie  and  Smith,  1977]  and  includes  roles  of  both 
a  syntactic  as  well  as  a  semantic  nature.  For  instance,  it  covers  such 
syntactically  defined  roles  as: 

— subject  of  transitive  verb  (agent-controller) 

( e . g .  John  hit  Mary.) 

— subject  of  transitive  (non-agent) 

(e.g.  John  received  a  telegram.) 

— object-complement 

(e.g.  They  elected  John  pres ident . ) 

— agent  of  passive 

(e.g.  Mary  was  hit  by  John. ) 

— concessive 

(e.g.  They  had  fun  .in  spite  of  John.) 

It  covers  such  semantic  role  as: 

— benef active 

(e.g.  They  wanted  to  win  the  game  for  John.) 

— circumstantial 

(e.g.  John  went  out  without  his  umbrella.) 
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— reason 

(e.g.  Mary  lost  her  job  because  of  John.) 


— price 

(e.g.  John  bought  the  watch  for  f ive  dollars.) 

— function 

(e.g.  John  used  the  baseball  bat  aj.  a  cane .) 

It  also  includes  such  locational  roles  as: 

— interior/position 

(e.g.  John  was  in  the  bathroom.) 

— proximate /destination 

(e.g.  John  went  up  to  the  door.) 

— superior /origin 

(e.g.  The  plane  dove  down  from  above  the  clouds.) 

— medial /referencial 

(e.g.  The  taxi  drove  between  the  two  pedestrians. ) 

— surf ace /parallel 

(e.g.  John  ran  his  hand  along  the  wall.) 

In  addition,  each  of  the  roles  was  viewed  in  three  to  five 

different  syntactic  contexts,  in  part,  to  uncover  different  strategies 
for  marking  the  same  relation  and,  in  part,  to  better  understand  the 
nature  of  the  relation  itself.  For  instance,  in  analysing  the  roles  of 
subjects  as  in 

— The  hunters  left. 

— The  lions  roared. 

it  is  often  enlightening  to  compare  their  behavior  in  different 
constructions.  In  this  case,  by  comparing  the  gerundives  of  these 

clauses,  i.e., 

— the  hunters'  leaving 
— *the  leaving  of  the  hunters 

— the  lions'  roaring 
— the  roaring  of  the  lions 

there  comes  to  light  a  variation  in  the  behavior  of  the  subject  noun 

phrases  which  may  well  be  a  result  of  a  difference  in  the  roles  which 

they  fulfill.  Here  "the  hunters”  are,  perhaps,  non-causal  agents  while 
"the  lions"  are  causal  agents.  In  other  cases,  variation  of  context  may 
lead  to  a  reinterpretation  of  the  role  being  examined.  For  example,  the 
preposition  "for"  in 
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— John  bought  the  book  for  five  dollars. 

is  often  said  to  mark  a  price  relation.  However,  given  that  price  is 
most  certainly  a  property  of  objects  and  that  properties  of  objects  can 
be  explicitly  stated  by  way  of  the  copular  construction,  the 
ungrammat i cal ity  of 

•It  is  for  five  dollars. 

comes  as  somewhat  of  a  surprise.  As  it  turns  out,  wliat  was  at  first 

thought  to  be  a  price  relation  is,  in  fact,  a  more  general  relation  of 

exchange  which,  being  impossible  to  interpret  as  a  property,  explains 

the  ungrammatical  copular. 

During  the  summer,  the  survey  was  administered  to  native  speakers 
of  Spanish,  Basque  and  Mandarin  and  the  results  reviewed  in  terms  of  the 
accuracy  of  the  translation  equivalences  and  the  awkwardness  of  the 
target  language  constructions.  It  should  be  pointed  out  that,  in 

addition  to  an  expected  handful  of  poorly  constructed  source  language 
items,  there  are  some  serious  inadequacies  in  the  survey.  For  one, 
because  of  the  size  of  the  survey,  items  pertaining  to  temporal  role 
relations  were  not  included.  For  another,  because  it  is  virtually 
impossible  to  tell  beforehand  which  roles  are  going  to  be  impossible  in 
which  contexts,  many  items  were  included  in  the  survey  despite  being 
ungrammatical  or  marginal  in  the  source  language.  The  result  was  that 
confusion  arose  on  the  part  of  the  informants  as  to  precisely  what  was 
being  investigated.  Finally,  the  survey  is  perhaps  inefficiently  long  as 
it  now  stands.  As  I  am  planning  on  collecting  data  from  some  ten  oi  so 
languages  altogether,  the  revision  and  streamlining  of  the  survey  will 
most  likely  be  required. 

2.  Preliminary  results  from  the  analysis  of  four  languages 

Across  the  board  I  found  evidence  for  the  motivation  of  three 
linguistic  classes  of  role  relations: 

—  syntactic :  roles  necessary  for  judging  syntactic  well-formedness 
(e.g.,  the  role  of  "subject"  is  necessary  in  order  to  understand  why 
"Is  raining."  is  not  a  grammatical  sentence  in  English). 

—  morphological :  roles  necessary  for  judging  the  morphological  well- 
formedness  of  constituents  within  sentences  (e.g.,  in  Basque,  the  role 
of  "means"  is  necessary  in  order  to  determine  that  "auto-an"(in  the 
car)  is  inappropriate  in  "Ni  autoan  noa.”  (I  am  going  by  car)). 

—  seaant ic :  roles  necessary  for  judging  the  well-formedness  of 
propositions  (e.g.,  the  role  of  "location"  is  necessary  in  order  to 
determine  that  "Juan  puso  el  libro"  (John  put  the  book)  is  ill-formed 
in  Spani sh) . 

Each  of  the  above  sets  of  relations  relies  on  different  marking 
strategies  and  all  three  interdependently  contribute  to  or  derive  from 
the  cognitive  representation  of  the  event  described. 
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The  syntactic  roles  motivated  by  the  languages  surveyed  are: 
subject,  primary  (direct)  object,  secondary  (indirect)  object,  and 
oblique  object.  Some  of  these  roles  are  exempified  in 

— [Eng]  They  gave  Mary  a  present. 

— [Spn]  (Elios)  le  dieron  un  regalo  a  Maria. 

(they)  her-i.o.  gave-they  a  present  to  Mary 

— [Bas]  (Haiek)  Mireni  oparia  eman  zioten. 

(they-erg)  Mary-dat  present-abs  given  it-have- 

her-they 

— [Man]  Tamen  gei-le  Mali  yi  ge  liwu. 

they  give-past  Mary  one  meas.  gift 

which  contain  subjects,  primary  objects  and  secondary  objects.  In 
English,  one  of  the  major  marking  strategies  is  word  order;  subjects 
precede  the  verb  while  objects  follow  it.  The  other  major  marking 
strategy  is  the  use  of  prepositions;  oblique  objects  are  preceded  by 
prepositions.  The  indirect  object  is  marked  either  by  word  order  (it  may 
precede  the  direct  object  as  in  the  example  above),  or  by  preposition 
whenever  it  does  not  precede  the  direct  object.  In  Spanish,  one  of  the 
major  marking  strategies  is  noun-verb  agreement;  subjects  determine  verb 
conjugations,  objects  determine  the  form  of  direct  and  indirect  object 
clitics  (i.e.  pronominals  like  "le"  in  the  example  above  which  are 
obligatorily  juxtaposed  with  verbs).  The  other  major  marking  strategy, 
as  in  English,  is  the  use  of  prepositions;  oblique  objects  are  preceded 
by  prepositions.  In  Basque,  the  major  marking  strategy  is  again  noun¬ 
verb  agreement;  subjects,  objects  and  indirect  object  all  play  a  role  in 
determining  the  form  of  the  verb.  In  Chinese,  one  of  major  marking 
strategies  is  again  word  order;  subjects  precede  the  verb,  objects 
follow  the  verb,  indirect  objects  precede  direct  objects  as  in  the 
example  above.  The  other  major  marking  strategy  is  the  use  of  pre-verbs; 
oblique  objects  are  preceded  by  pre-verbs. 

The  major  marking  strategies  for  syntactic  roles,  then,  are: 

— word  order 

— noun-verb  agreement 

— prepositions/pre-verbs 

while  the  minor  marking  strategies,  which  were  not  discussed,  are: 

— bound  affixes  (e.g.  Harry'  £  horsing  around) 

— morphophonemic  alternations  (e.g.  I  saw  them.  They  saw  me . ) 

— derivational  rules  (e.g.  subjects  of  gerunds  undergo  N  >  ADJ :  My 
leaving  so  early) 
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The  lexical  roles  distinguished  fall  into  two  general  classes: 
event  related  and  location  related.  Schematically  the  event  related 
roles  include: 

relation  marker 


English 

Spanish 

Basque 

Mandarin 

agent-actor 

— 

— 

-k 

— 

patient 

— 

— 

-0 

ba 

source 

f  rom 

de 

-tik 

cong 

possessor 

of/  's 

de 

-en 

de 

property 

of 

de 

-tako 

you 

mode 

wi  th 

con 

-z 

— 

instrument 

with 

con 

-z 

yong 

accompaniment 

with 

con 

-ekin 

gen 

causer 

by/ 

por/ 

-gatik 

be  i 

because 

of  a  causo 

de 

benefactor 

for 

por 

-tzat 

wei 

purpose 

for 

para 

-rako 

y ingwe i 

reference 

about 

sobre 

buruz 

— 

goal 

to 

a 

-i 

gei 

while  the  location  related  roles  are: 


relation 

Engl ish 

marker-- 

Spanish 

Basque 

destination 

to 

a 

-ra 

limitation 

up  to 

hasta 

-raino 

direct  ion 

toward 

hacia 

-runtz 

origin 

from 

desde 

-tik 

position 

at 

en 

-an 

contact 

against 

contra 

-an 

interior 

in 

dentro  de 

ba  rru-an 

into 

a 

barru-ra 

through 

por 

-tik 

exterior 

outside 

fuera  de 

kampo-an 

out  of 

de 

-tik 

anterior 

before 

delante  de 

aurre-an 

posterior 

behind 

detras  de 

a  tze-an 

superior 

on 

encima  de 

g 3 in-ean 

inferior 

under 

debajo  de 

azpi-an 

lateral 

beside 

al  lado  de 

ondo-an 

by/past 

al  lado  de 

alde-an 

circumfer. 

around 

alrededor  de 

inguru-an 

medial 

be  tween/ 

entre 

-en  arte-tik 

parallel 

amoung 

along 

a  lo  largo 

-tik 

perpendic . 

across 

a  traves  de 

-an  zehar 

ul terior 

beyond 

mas  alia  de 

haruntz-ago 
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The  major  strategy  for  marking  lexical  roles  is,  unsurprisingly, 
through  the  use  of  lexical  or  morphological  items.  In  English,  it  is  by 
way  of  prepositions  for  all  roles  other  than  those  related  to  the  direct 
participants  in  the  event  expressed  by  the  verb:  agent-actor,  patient, 
experiencer.  The  same  is  true  of  Spanish.  In  Basque,  the  marking  of 
lexical  roles  is  for  the  most  part  by  way  of  case  markers  in  the  form  of 
bound  suffixes.  However,  in  the  case  of  the  direct  participants,  the 
language  relies  redundantly  on  noun-verb  agreement  while  for  the 
orientationally  determined  locational  roles,  postpositions  are  called 
upon.  In  Chinese,  the  lexical  roles  are  marked  by  pre-verbs. 

The  semantic  roles  which  have  been  tentatively  identified  are: 
actor,  theme,  goal,  source,  location,  time,  manner,  and  cause.  These 
roles  are  marked  by  way  of  a  variety  of  syntactic,  lexical  and 
morphological  devices  many  of  which  are,  at  present,  not  well 
understood . 

B.  A  summary  of  the  results  from  Part  B 

The  following  is  largely  derived  from  a  description  by  John  Lyons  of  a 
theory  of  linguistics  advanced  by  L.  Tesniere  which  is  based  on  event 
analysis  [Lyons,  1978].  Any  situation  can  be  described  as  either  a  state 
or  a  happening.  Happenings  can  be  categorized  as  processes  or  events, 
which, in  turn,  can  be  subclassified  in  terms  of  doings  or  goings-on.  States 
require  the  presence  of  some  neutral  entity  (unmarked  for  role)  but,  beyond 
that,  may  or  may  not  possess  such  circumstancial  roles  as  time  or  place  or 
the  roles  of  attribute  or  class.  Happenings,  on  the  other  hand,  will 
variously  require  such  participant  roles  as  agent,  patient,  goal  and  source 
as  well  as  the  circumstancial  roles  mentioned  above  along  with  the  roles  of 
cause  and  effect.  Unfortunately,  the  motivations  for  the  roles  in  this 
system  are  in  many  ways  linguistic  and  there  remains  a  good  deal  more 
review  before  a  solid  proposal  can  be  made. 

With  respect  to  inferencing,  perhaps  the  most  effective  systems  to 
date  are  those  based  on  Prof.  R.  Schank's  "conceptual  dependency" 
representations  [Schank  and  Riesbeck,  1981],  The  basic  set  of  primitive 
role  relations  posited  include:  actor,  object,  goal  and  source.  Roles  such 
as  instrument,  cause  and  effect  are  captured  by  way  of  dependencies  between 
CDs.  Again  the  motivation  for  the  roles  is  to  some  extent  determined 
linguistically;  the  actor  and  object  slots  in  a  CD,  for  instance,  are  more 
motivated  by  the  existence  of  the  grammatical  relations  of  subject  and 
object  than  by  any  justification  in  terms  of  event  analysis.  Bat,  for  the 
most  part,  the  roles  are  determined  on  the  basis  of  the  kinds  of  inferences 
which  the  systems  is  meant  to  reason  about,  namely,  causal  connections. 
Again,  much  material  remains  to  be  reviewed  before  a  solid  proposal  can  be 
made . 


10 


Data  from  Mandarin  has  yet  to  be  analysed. 
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2«  Encoding  the  Natural  Language  Meaning  of  Time— 
7.1.  Introduc  t ion 


This  paper  is  more  in  the  nature  of  a  progress  report  than  a  treatise  on 
time;  it  offers  certain  conclusions  and  insights  stemming  from  our  recent  work 
in  the  language-universal s  and  cognitive-universal s  of  time.  We  have  been 
concerned  with  understanding  encoding  strategies  for  time  and  why  they  are 
manifestly  different  from  language  to  language.  This  investigation  on  encoding 
strategies  has  covered  a  variety  of  unrelated  languages,  including  English, 
and  by  virtue  of  the  fundamental  association  between  space  and  time,  dealing 
also  with  spatial  meaning  has  been  inevitable.  This  functional  interdependence 
between  time  and  space  in  perception  and  language  has  been  long  recognized 
both  in  psychology  [Miller  and  Johnson-Laird,  1976;  Piaget,  1956,  1964,  1969], 
and  in  linguistics  [Lyons,  1978;  Anderson,  1971],  Although  this  intimate 
relationship  between  space  and  time  is  a  phenomenon  of  fundamental  importance 
for  a  general  theory  of  cognition  and  language,  our  reasons  for  studying  the 
encoding  strategy  of  time  and  space  are  quite  different.  We  are  primarily 
interested  in  designing  and  implementing  computer  programs  that  will  be 
ultimately  capable  of  understanding  natural  language  meaning,  and  for  this 
purpose  we  need  to  develop  the  ability  to  deal  with  the  meaning  of  time  and 
space  as  they  are  encoded  in  natural  language.  We  are  also  interested  in 
developing  a  conceptual  representation  of  time  and  space  that  is  sufficiently 
general  as  to  be  applicable  successfully  to  English  as  well  as  other  natural 
language  systems  such  as  Burmese,  Chinese,  Cree  or  any  other  language. 

In  this  paper  our  primary  goals  are  to  present  a  general  conceptual  model 
of  time,  and  to  discuss  the  consequences  of  viewing  the  NL  meaning  of  events. 
event-interv".! ,  and  interval  relationships  in  terms  of  that  model  .  We  do  not 
vet  know  the  full  range  of  implications  attendant  upon  a  model  such  as  this, 
nor  are  we  in  a  position  to  claim  that  these  universal  concepts  of  time  are 
appropriate  as  bases  for  representing  time.  We  have  only  reached  the  stage 
where  we  can  suggest  as  universal  a  list  of  concepts  of  time,  and  a  .  ■  ntativ 
plan  for  using  this  list  as  basic  material  for  representing  tir’*>  ;n  Lateral 
language . 

2.2.  The  Universal  Model  of  Temporal  Meaning 

The  most  basic  role  of  time  lies  in  understanding  the  meaning  of  events 
in  the  real  world.  We  understand  events  in  terms  of  their  duration,  serial 
order  of  occurrence,  overlappings,  the  aspects  of  beginning  and  concluding, 
and  so  forth,  all  intrinsically  temporal  concepts.  The  categories  presented 
here  and  the  conceptual  classif icatory  systems  proposed  are  all  represented  in 
all  the  languages  surveyed.  We  shall  refer  to  the  natural  language  categories 
of  the  meaning  of  time  as  natural  language  metaphors. 


^Submitted  to  the  Inter-Disciplinary  Workshop  on  Motion: 
and  Perception,  Toronto,  April  4-6,  1983. 
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2.2.1.  Natnral  Language  Metaphors 

Natural  language  metaphors  for  real-time  meaning  may  be  organized  under 
three  headings. 

2-2. 1.1.  Natural  Language  Metaphors  for  Event  Types 

The  distinction  that  is  conceptually  most  basic  within  the  natural 
language  meaning  of  time  is  the  contrast  that  exists  between  those  events  that 
are  processes  in  time,  as  against  those  that  exist  as  states.  This  is  the 
process  vs.  state  categorical  distinction;  all  natural  phenomena  occur  either 
as  a  process-type  or  as  a  state-type. 

The  second  most  basic  distinction  of  the  universal  model  of  time  is  the 
inherent  duration-type  contrast;  a  process-type  event,  or  a  process  that  is  in 
reality  a  state  in  flux,  may  have  a  duration  length  that  is  either  durative  or 
punctual . 

In  English,  the  duration  span  of  events  related  to  the  word-sense  of 
verbs  such  as  "hit,  blink,  kick,  smash,  etc."  is  considered  punctual ;  notice 
that  because  it  is  part  of  our  real  world  knowledge  that  these  durations  are 
brief  and  momentary,  when  we  use  these  verbs  in  the  present  progressive  tense, 
as  in  "John  is  blinking  his  eyes,"  the  meaning  imparted  is  one  of  repetition 
of  the  act  "blink"  and  not  the  protraction  of  a  single  act.  The  default 
meaning  concerning  repetition  corresponds  to  our  stored  knowledge  about  the 
type  of  events  encoded  into  these  word-senses. 

The  verbs,  "siphon,  bathe,  perform,  swim,  drive,  etc."  entail  durative 
spans  or  intervals  when  realized  in  real  time.  In  the  expression  "John  is 
bathing,"  we  do  not  infer  a  series  of  repeated  acts  of  "bathe."  The  act  "kill" 
seems  to  embody  in  its  word-sense  inventory  both  of  the  properties  of  the 
punctual  and  durative  meanings.  For  instance,  in 

(1)  John  is.  killing  me, 

the  present  progressive  tense  .is  killing  entails  a  meaning  for  kill  where  the 
agent  (John)  is  doing  things  that  may  eventually  bring  about  the  effect  of 
"kill."  In  this  case,  "kill"  is  durative.  However,  in 

(2)  Look,  John  _i£  killing  rats  in  the  alley. 

"kill"  seems  to  be  a  punctual  verb  and  the  repetition  of  a  series  of  punctual 
acts  seems  to  be  the  most  natural  reading. 

This  probably  suggests  that  both  "punctual  and  durative"  time  values  are 
listed  as  alternates  in  the  word-sense  definition  for  "kill.”  It  is 
interesting  to  note  that  in  Burmese  and  J inghpaw  the  word  for  "kill"  is  used 
only  in  the  punctual  sense  of  time  duration. 

2.2.1 .2.  Natural  Language  Metaphors  for  Event  Orientation:  Aspect 


Aspectual  distinctions  add  two  conceptual  organizers  to  the  available 
categories'  boundary  reference  system  (onset  and  terminal)  and  the  values 
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indicating  the  state  of  time  flow,  (progressive  and  complet ive) .  The  boundary 
reference  categories  enable  the  perception  of  a  given  event  with  reference 
either  to  its  beginning  (inception  in  the  case  of  the  process  of  state 
change),  or  to  its  ending.  The  tine-flow  categorical  distinction  enables  us  to 
distinguish  events  in  progress  from  those  that  have  been  completed. 

Examples : 

(1)  Onset-progressive  a  spec  t .  process -durat ive  verb 
"John  is  starting  to  siphon  gas  from  your  Volvo." 

(2)  Onset-completive,  process-duntive  verb 

"John  has  started  to  siphon  gas  from  your  Volvo." 

The  "onset"  aspect  is  still  being  effected  in  (1),  but  has  already  been 
accomplished  in  (2). 

(3)  Onset-progressive,  process-punctual  verb 
"John  is  starting  to  blink  the  headlights." 

(4)  Onset-completive,  process-punctual  verb 

"John  has  (already)  started  blinking  the  headlights." 

(5)  Terminal-progressive,  prece s s-punc tua 1  v e rb 

12 

"John  is  stopping  blinking  the  headlights  now." 

(6)  Terminal -completive,  process-punctual  verb 
"John  has  stopped  blinking  the  headlights." 


etc . 

7 .2 .1^.3  .  Natural  Language  Metaphors  for  the  Speakers  Point  of  View:  Tense 

Natural  language  encoding  of  the  temporal  aspects  of  event  meaning 
provides  three  categorically  distinct  vantage  points  from  which  the 
realization  of  given  events  is  recorded  and  regarded.  The  interplay  between 
the  speaker’s  position  and  the  relationship  it  has  to  an  event  are  the  crucial 
considerations  behind  this  distinction.  First,  the  speaker  may  place  himself 
as  an  on-scene  witness  of  the  event  that  is  unfolding,  and  regard  the  event  as 
happening  in  the  temporal  present.  Second,  the  speaker  may  retrospectively 
recall  an  event  and  give  an  account  to  it  as  having  happened  in  a  time  frame 
already  past.  Third,  the  speaker  may  speculate  and  conjecture  an  event  which 
has  not  yet  been  realized  in  terms  of  real  time. 


The  meaning  of  this  expression  can  be  characterized  as  a  script  consist¬ 
ing  of  all  the  physical  steps  necessary  to  execute  the  act  "stop  blinking"; 
hand-eye  coordination  to  locate  headlight  dial,  placement  of  hand,  taking  bold 
of  dial  with  thumb  and  first  two  fingers,  muscle  tone  to  turn  off  switch, 
etc.,  will  be  captured  by  this  expression.  The  moment  itself  when  blinking 
stops  will  be  a  punctual  event,  which  is  why  steps  leading  to  that  moment  are 
perceived  as  being  prior  to,  rather  than  during,  the  punctual  event  "stop." 
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The  first  tense  is  present .  the  second  past,  and  the  third  modal  in 
traditional  grammars.  However,  for  considerations  of  universal  application  we 
adopt  a  slightly  modified  version  of  the  Cree  distinction  and  will  refer  to 
the  tense  categories  as  WITNESS,  RETROSPECTIVE  and  MODAL.  A  number  of 
reasoning  exists  for  not  retaining  the  traditional  terms  of  the  present,  past, 
and  fntnre  tenses .  Although  one  may  deem  these  terms  appropriate  for  English, 
when  considered  against  the  requirements  of  a  universal  model  of  time  in 
natural  language,  there  are  serious  drawbacks.  These  terms  overlook,  for 
instance,  the  role  of  spatial  concepts  in  encoding  the  meaning  of  time  in 
natural  language;  in  the  vast  majority  of  languages,  mechanisms  for  spatial 
meaning  (e.g.  prepositions  at,  from,  up.  etc.)  play  crucial  parts  in  conveying 
the  meaning  of  time.  The  future  tense  is  also  problematic  because  it  tends  to 
conceal  basic  semantic  relations  and  syntactic  patterns.  For  instance,  "I  will 
come  on  Thursday"  and  "I  plan  to  come  on  Thursday"  yield  the  same  inference  as 
regards  to  the  speaker's  intention,  and  yet,  by  criteria  of  tense,  only  the 
former  is  syntactically  conceptually  modal  in  the  tense  value  specified, 
meaning  that  both  assertions  indicate  an  action  or  event  that  have  not  yet 
been  realized  in  real  time. 

Diagram  I  shows  the  conceptual  model  of  time  in  natural  language  that  we 
propose;  the  terminal  items  of  each  tree  represent  the  primary  concepts  of 
time  in  natural  language.  In  our  opinion  the  aspect-tense  meaning  of  time  is 
probably  adequate  in  its  present  form  as  a  model  of  the  kernel  concepts  of 
time;  the  modal  among  the  tense  concepts  requires  the  most  extensive  work  to 
be  done.  For  conceptual  as  well  as  semantic  reasons,  the  modal  tense  category 
embodies  the  future  tense,  potential  events  that  may  or  may  not  actually 
happen,  events  that  may  be  predictable  from  one's  knowledge  of  habits  of  a 
given  subject-actor,  obligations  and  compunctions,  etc.  In  other  words,  we 
have  grouped  together  as  a  semantic  domain  those  tense  values  which  do  not 
fall  into  the  witness  a^d  retro  tenses.  The  internal  configuration  of  this 
complex  rubric  is  being  explored. 

2.2. 1.4.  Conceptual  Representation  of  Time  in  NL:  An  Illustration 

Let  us  first  schematize  the  categories  of  lexically  encoded  information, 
and  then  the  grammatical,  and  finally,  the  conceptual  representation. 

In  Diagram  II,  we  give  an  illustration  of  our  ideas  for  representing  the 
meaning  of  time  in  the  verbs  siphon  and  hit .  The  concepts  of  time  —  aspect 
and  tense  —  interdigitate  with  those  pertinent  temporal  information  that  are 
part  of  the  word-sense  meaning  of  the  verbs  and  yield  a  series  of  plausible 
natural  language  expressions.  This  may  be  a  possible  approach  to  the 
representation  of  time  in  natural  language,  however  we  realize  that  we  have 
yet  to  investigate  the  ramifications  possible  and  implications  entailed  by  the 
approach. 

This  type  of  meaning  representation,  when  combined  with  the  sort  of 
case-role  information  that  the  Schankian  conceptual  dependency  provides,  can, 
we  believe,  become  a  rich  and  viable  system  to  capture  the  meaning  of  natural 
language. 
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2*2.1.. 5,  Implications  of  Conceptual  Representation  of  Time  in  Natural 
Language 

Some  general  remarks  may  be  made  as  regards  our  ideas  for  the  conceptual 
representation  of  time  just  outlined.  For  one  thing,  this  representation 
corresponds  to  the  fact  that  the  conceptual  and  natural  language  levels  have  a 
one  many  relationship.  A  single  specification  of  reference  boundary  aspect, 
say  [+onset,-term]  corresponds,  in  English,  to  the  semantic  behavior  of  a 
class  of  auxiliary  verbs  like  "begin,  start,  initiate,  commence,  get  to,  etc." 
In  this  relationship  the  higher  (conceptual)  level  governs  and  constrains  the 
lower  level,  the  lower  level  is  more  complex  vis-a-vis  the  higher  level,  etc. 
Further,  the  specification  [verb.+durative]  will  treat  all  the  word-seivses 
having  the  process-durat ive  values  in  the  lexicon  as  fulfilling  the  conditions 
of  the  specification.  The  natural  language  realization  of  conceptual  tense  is 
another  case  in  point;  [+RETRO] ,  for  instance,  will  induce  tense  inflexion  in 
most  verbs  and  assign  grammatical  auxiliaries  of  tense  in  the  ether  cases.  For 
instance,  "broke,  got,  was  mad,  were  spirited,"  are  in  RETRO  tense. 
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John  is  trying  to  save  money  for  Christmas. 


(4)  John  has  been  saving  money  for  Christmas. 

The  auxiliary  mechanism  "planning  to  save"  represents  a  meaning  of  time  where, 
in  one  reading,  John  is  in  the  "planning"  time  slot,  and  perhaps  not  yet  in 
the  actual  "save"  slot.  Therefore,  in  relation  to  the  "tense"  of  the  main  verb 
"save,"  John’s  temporal  location  at  the  time  of  speaking  is  MOD.  The  meaning 
of  "save"  is  thereby  rendered  a  potential  act,  an  intention,  rather  than  a 
realtime  event.  There  are  other  syntactic  approaches  to  this  meaning;  e.g. 
"John  will  be  saving."  Whether  in  this  instance  the  semantic  auxiliary  "plan” 
is  chosen,  or  whether  the  grammatical  auxiliary  "will"  is  picked,  similar 
inferences  will  be  obtained  concerning  the  time  value  of  MOD  tense  of  the  main 
verb  "save"  because  "plan"  and  "will"  have,  conceptually  identical  modifying 
influence  on  the  main  verb. 
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In  (4)  the  temporal  meaning  of  "save"  is  in  the  progressive  aspect,  and 
the  perfect  tense  "has  been"  indicates  that  John  has  been  at  it  for  a  long 
time.  This  is  the  persistive-progressive  aspect,  a  category  we  have  not  dealt 
with. 


The  point  that  we  wish  to  allude  is  this:  syntactic  mechanisms  may  vary 
if  they  are  associated  with  a  given  conceptual  function,  as  "will"  and  the 
auxiliary  "plan"  are  to  MOD. 

In  the  following  section  we  want  to  explore  the  natural  language 
description  of  events  by  using  the  conceptual  categories  we  now  have  to 
characterize  it. 

7 .3 .  The  Natural  Language  Description  of  Event  and  Event-Interval 

"Events  happen,  states  do  not  ...  the  only  events  that  can  happen  are 
those  identified  perceptually  as  occurring  as  some  location  during  some  brief 
span  of  time"  according  to  Miller  and  Johnson-Laird  [1976].  While  we  will  not 
be  dealing  with  the  problem  of  trying  to  define  the  perceptual  meaning  of 
event  in  its  full  setting  (for  example,  see  Waltz  [1982])  we  will  be 
interested,  however,  in  developing  a  framework  for  dealing  with  it  in  the 
sense  of  its  duration  and  in  terms  of  its  role  in  understanding  motion.  We 
will  draw  illustrative  examples  from  English  in  developing  a  conceptual  model 
for  understanding  "event."  However,  since  we  are  approaching  the  natural 
language  meaning  of  "event"  and  "motion"  only  from  the  temporal  perspective, 
our  characterizations  cannot  possibly  cover  all  the  essential  aspects.  For 
instance,  we  have  no  way  of  knowing  whether  our  approach  can  be  conveniently 
linked  up  with  conceptual  dependency  accounts  of  case-role  information,  and 
yet,  it  is  clear  to  us  that  a  full  meaning  representation  of  event/motion  will 
not  be  possible  until  such  a  linkage  has  been  achieved,  and  that  will  require 
far  more  than  a  preliminary  suggestion. 

Conceptually  and  linguistically,  there  are  two  interlocking  components  in 
specifying  event,  the  kernel  and  the  shell  components.  The  kernel  component 
specifies  the  basic  or  core  meaning  of  event,  whereas  the  shell  component 
contains  a  series  of  modifiers  of  the  nuclear  meaning  of  event.  In  specifying 
the  nucleus  of  the  meaning  of  event  the  kernel  component  provides  the 
linguistic-cognitive  universal  conceptual  categories  of  time  that  we  discussed 
in  the  preceding  chapter.  The  shell  component  features  modifiers  which  specify 
special  conditions  on  the  interpretation  of  the  nuclear  meaning  of  event.  The 
following  schematic  constructs  illustrate  the  components  of  event  meaning. 

Roughly,  the  kernel  component  can  be  thought  of  as  where  "practical  and 
lexical  knowledges"  [Miller,  1977]  come  together,  resulting  in  lexical 
selection:  a  verb  whose  word-sense  matches  a  perceptual  event  to  oversimplify 
it  of  course,  is  selected  to  characterize  that  event  verbally.  A  given  word- 
sense  meaning  (state/process:  dur ative/punctual  process)  is  initially 
specified  with  the  selection  of  a  verb.  Next,  presumably  grammatical 
assignment  of  aspectual  meaning  is  accomplished,  and  additional  structure  is 
thereby  acquired  by  the  core  meaning  of  event;  boundary  orientation 
(onset/terminal),  and  flow-state  description  (progressive/completive),  now 
provide  a  hypothetically  adequate  basis  for  characterizing  the  event  (see 
Diagram  II).  Farther,  the  grammatical  tense  meaning  provides  communicative 
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salience  by  channeling  event  meaning  through  one  of  three  possible 
perspectives  (speaker  as  on-scene  witness,  speaker  in  retrospective  mood,  or 
speaker  as  speculating  the  potential  and  possible  happening,  whether  already 
realized  or  not  of  an  event). 

These  items  of  information  provide  the  basic  milieu  of  event  meaning. 
Should  there  be  reason  for  modifying  in  some  way  the  nuclear  conf iguration 
(see  Diagram  II),  the  shell  component  will  provide  the  mechanism.  To  show  the 
domains  of  conceptual  and  semantic  functions  involved  in  specifying  the 
primary  meaning  of  event  in  time  the  following  schenatization  maj  be  made. 

In  terms  of  natural  language  correlates,  the  shell  component  contains 
such  modifiers,  auxiliaries,  and  other  time  expressions,  many  of  which  are  in 
the  form  of  adverbial  expressions.  We  shall  give  illustrations  of  these  within 
a  taxonomic  schema  of  modifiers  of  basic  event  meaning. 

The  shell  component  is  made  up  of  two  basic  types  of  modifiers  of  the 
basic  meaning  of  event,  the  adverbial  modifier  system  and  the  auxiliary 
modifier  system.  The  distinction  is  overtly  syntactical  but  priraariiv  a 
semantic-conceptual  distinction.  The  shell  functions  seem  to  form  a  natural 
taxonomy  in  the  follovfing  way. 

7.3..1.  Locative  Reference  Time 

This  modifier  function  may  be  manifested  in  the  surface  syntactic 
configuration  as  a  prepositional  expression  (see  (la),  below)  or  ar,  time 
adverb  (see  below) . 

(la)  I  saw  Ambrose  at  noon. 

Here,  "at  noon"  is  a  locative  reference  point  of  t ime  realized  as  a 
prepositional  expression,  as  is  also  the  case  with  "in  1808,"  next. 

(lb)  Franz  Joseph  Haydn  died  i_n  1809 . 

(2a)  Ambrose  was  building  superintendent  here  f roa  Way  1964  to  June  1981 . 

(2b)  I  waited  for  Ambrose  from  noon  till  six  today. 

In  (2a, b)  the  locative  reference  item  is  specified  in  the  form  of  duration 
intervals.  Locative  reference  time  provides  the  basic  description  of  event 
with  concrete  landmark  reference. 

2.3.2.  Auxiliary  Modif i ers 

The  auxiliary  modifer  function  of  time  in  the  shall  component  nay  be  a. 
grammatical  form,  as  in  "will  begin  to  broadcast,"  or  a  verbal-semant ic  form, 
as  in  "expect  to  begin  to  broadcast."  The  typology  of  auxiliary  modifier 
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13 

functions  that  we  have  adopted  is  based  upon  the  interaction  pattern  between 
the  tine  value  of  the  main  verb  (MV)  and  the  auxiliary  (AUX) .  There  are  three 
clear  patterns. 

(3)  John  is  trying  to  get  home  for  Christmas. 

The  time  value  of  the  auxiliary  "try"  is  a  functional  part  of  the 
temporal  value  of  the  main  verb  "get,"  and  the  direct  witness  (present) 
tense  meaning  indicates  that  this  temporal  relationship  between  the 
auxiliary  and  the  main  verb  is  a  real-time  interaction.  John  is,  in  fact, 
already  engaged  in  activities  that  will  contribute  directly  to  his  being 
able  to  "get  home  for  Christmas."  The  auxiliary  has  no  time  value  that  is 
independent  from  the  value  specified  for  the  main  verb. 

Now  consider  the  next  example. 

(4)  John  plans/ intends/hopes/expects  to  get  home  for  Christmas. 

The  time  value  of  the  auxiliary  here  cannot  be  taken  as  being  within  the  real 
time  frame  of  the  main  verb;  in  fact,  the  meaning  of  "expect,  hope,  intend, 
..."  may  be  in  the  nature  of  mental  states,  and  not  real-time  actions.  The 
interaction  between  this  type  of  auxiliary  and  the  time  valueof  the  main  verb 
is  modal :  in  other  words,  if  John  occupies  at  the  time  of  speaking,  the 
"intend,  plan,  hope  ..."  frame,  then  "get  home"  is  really  no  more  than  a 
declared  goal.  Hence  the  modal  pattern. 

The  grammatical  auxiliary  will/would.  may/might.  etc.,  behave 
correspodingly  in  their  interaction  with  the  time  value  of  the  main  verb. 

Finally,  the  auxiliary  "suppose"  deserves  a  special  mention;  we  have 
designated  this  modifier  as  modulatory  because  in  addition  to  modifying  the 
time  value  of  the  main  verb,  it  also  introduces  other  factors.  Although  it  is 

not  a  modifier  of  time  per  se .  as  the  other  auxiliaries  are,  it  is 

nevertheless  included  for  the  reason  that  "suppose"  does  not  appear  to  allow  a 
witness  tense  reading,  as  may  be  observed  in  the  next  examples. 

(5)  I  suppose  that  I  will  fly  to  Seattle,  (a  MOD  tense) 

(6)  I  suppose  that  I  did  fly  to  Seattle  that  time,  (a  RETRO  tense) 

(7)  I  suppose  that  I  fly  to  Seattle,  (a  direct  WIT  tense  —  ambiguous 

meaning) 

(8)  I  suppose  I  am  flying  to  Seattle,  (a  MOD  tense  inference) 

The  pattern  to  notice  here  is  the  impossibility  of  a  direct  witness  tense,  as 
in  (7),  showing  that  there  is  a  tense  constraint  associated  with  the  modifying 
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function  of  the  auxiliary  "suppose";  it  can  function  only  in  the  modal  and 
retro  tense  environments.  In  psychological  terms,  these  tense  types  are 
"mental  time,"  as  against  the  real-time  of  the  witness  tense.  For  instance  (7) 
is  completely  acceptable  if  the  intended  reading  is  that  one,  a  pilot  for 
instance,  regularly  flies  to  Seattle.  The  point  to  note  is  that  regularity  or 
habituality  is  not  in  itself  a  real-time  event,  and  hence  not  a  WIT  tense,  but 
rather  a  MOD  tense. 

The  auxiliary  "suppose"  modifies  more  than  the  time  value  of  the  main 
verb,  it  changes  the  color  of  in  tent i ona 1  i  ty  altogether.  However,  this  is 
beyond  the  scope  of  our  present  concern. 

A  schematic  for  the  shell  component  of  event  tray  be  given  as  follows. 

2. 3.-1.  Iterative  Modifiers 

A  special  variety  of  adverbial  noii'ers  of  time  are  iterative  functions 
which  provide  the  logical  mans  for  describing  repetitive  occurrences  in  time, 
including  linear  order  of  succession  of  events,  and  the  various  manners  of 
overlapping  relationship  between  grups  of  events.  Some  examples  cf  these  are 
given  ir  the  following. 

Repetition  £f  a_c_t  i_n  t  ime 

(9)  Every  morning ,  Ambrose  would  stand  by  the  door  and  greet  you,  day  in  end 
day  out,  year  after  year.,  until  he  retired. 

(10)  See  how  John  hcyt>  hitting  her,  again  and  again? 

(11)  We  hold  this  family  reunion  annual lv. 

Order  of  succession  in  tying? 

(12)  First  John  came,  then  Bill  arrived,  then  Jill,  then  Tim,  t_he_n  Sophie  .... 

(13)  Sophie  was  last  to  get  here,  and  Tim  before  her,  Jill  before  Tin,  Bill 
before  Jill  and  John  before  everybody. 

(14)  John  was  first,  Eill  second,  Jill  third,  Tim  next  and  Sophie  last. 

It  is  obvious  that  specifying  the  order  of  succession,  or  linear  sequencing, 
follows  procedures  based  upon  the  principle  of  ordinal  relationships;  and 
since  ordinal  numbering  is  a  logical  process,  counting  may  be  flip-flopped  to 
suit  the  need  of  the  speaker,  as  indicated  in  (13)  and  (14),  especially 
clearly. 

The  natural  language  description  of  event  sequences  may  refer  to  boundary 
landmark  references,  as  in 


ONE  Progress  Eeport 


-32- 


L.  Karan,  D.  Spoor  sad  D.  Waits 


(15)  We  are  trying  to  start  our  journey  to  Tibet  but  Bill  over  there  oust 
first  finish  up  his  packing. 

or  to  temporal-flow  aspect,  as  in 

(16)  We  are  trying  to  start  our  journey  but  Bill  there  i_s  still  packing. 

Also 


(17)  We  have  finished  preparation  for  starting  except  for  Bill  who's  still 
packing. 

Facets  of  aspectual  meaning  of  events  display  sequencing  relationships  in 
these  examples.  Sequencing  and  overlapping  relations  concern  only  multiple 
events. 

Overlapping  intervals 

The  natural  language  description  of  interval  relationship  that  overlap 
employs  such  function  words  as  "during,  while,  through,  when  ...."  These 
descriptions  may  present  two  intervals  without  reference  to  boundaries,  as  in 


(18)  While  I  was  watching  TV  Tom  was  doing  his  homework. 

(19)  When  I  was  watching  TV  Too  was  doing  his  homework. 

In  principle,  these  descriptions  imply  that  there  was  full  durational 
overlapping  between  the  two  intervals.  Partial  overlap  is  described  by  means 
of  encoded  reference  to  event  boundaries. 

(20)  I  was  arriving  at  the  party  but  Tom  was  leaving. 

Here,  "arrive"  and  "leave"  are  two  separate  intervals  which  may  well  overlap 
fully,  but  each  overlaps  with  the  interval  "party"  only  partially. 
Presumably,  one  never  occupies  an  entire  "party"  interval  trying  to  "arrive" 
or  trying  to  "leave"  it.  Since  the  span  of  the  interval  "part y"  is  far  longer 
than  the  other  two  intervals  only  partial  overlap  will  be  possible  in  real 
time. 


The  natural  language  meaning  of  overlap  can  reflect  perceptual  bias,  as 
in  the  next  example. 


(21)  When  the  Brooklyn  Bridge  collapsed.  Bill  had  just  entered  it  from  Lower 
Manhattan  and  Bob  from  Brooklyn  Heights. 

The  overlap,  i.e.  temporal  simultaneity,  between  Bill's  approaching  the 
bridge  and  the  latter's  collapse  is  apparent;  furthermore,  the  simultaneity 
between  Bob's  approaching  the  bridge  and  the  time  of  collapse  of  the  bridge  is 
also  highly  apparent.  However,  what  may  not  be  equally  apparent  is  the  fact 
that  the  actions  of  Bill  and  Bob  also  overlap  in  that  they  are  simultaneous 
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events,  a  relationship  that  could  be  inferred  by  the  transitive  nature  of 

these  events. 

(22)  I  can't  leave  until  Joe  gets  here . 

(23)  Now  that  Joe's  here  I  am  leaving. 

The  overlapping  relationship  between  intervals  represented  by  oriented  motion 
verbs  (cone/go,  arrive/leave)  is  particularly  interesting  because  it 
introduces  reference  boundary  aspects  that  are  internal  to  the  structures  of 
events  in  qu  stion.  Consider  (23)  in  terms  of  aspectual  details  now. 

(24)  A.  At  tli e  time  of  speaking  -  Joe  is  here 

I  am  also  here 

B.  But  Joe's  being  her?  is  part  of  his  act  (interval)  of  "arriving," 
whereas  my  being  here  is  part  of  my  act  (interval)  of  "leaving." 

C.  Surely,  my  act  of  "leaving"  has  a  longer  interval  span  than  the  span 
of  up  being  at  the  sa-e  location  with  Joe  momentarily;  presumably, 

my  leaving  this  place  is  integral  to  ray  going  sons  place  else.  In  the 
sn me  sense,  Joe's  "arriving"  interval  is  part  of  his  larger  interval 
"come." 

D.  In  fact,  without  the  aspectual  meaning  of  tin?  which  provides  inter¬ 
nal  structure  to  event  intervals,  it  will  be  extremely  difficult,  if 
not  impossible,  to  understand  the  natural  language  meaning  of  rela¬ 
tions  such  as  (23). 

2 .?  .4.  Understanding  Interval  R  ?  1  ajti  on  shin  s 
Let  us  return  to  example  (.14)  . 

(14)  John  was  first.  Bill  was  second,  Jill  third.  Tin  next  and  Sophie  last. 

Waat  do  the  ordinal  terms,  and  "next,  last"  all  mean?  This  utterance  was  made 
in  the  context  of  describing  the  order  of  arrival,  in  real  time,  of  the 
persons  named.  In  understanding  natural  language  descriptions  such  as  this  we 
must  surely  have  access  to  a  conceptual  representation  where  the  ordinals 
stand  for  event-intervals  for  "arrive";  now  then,  it  follows  that,  the 
ordinals  have  time  value  as  stand-in  for  actions  which  command  by  their 
nature,  duration  spans.  The  linguistic  stand-in  for  the  time  intervals,  the 
ordinals,  are  from  the  standpoint  of  understanding  natural  language, 
associated  with  default  values  of  time  which  are  specified  in  the  word-sense 
of  "arrive."  Without  a  model  of  time  that  allows  robust  interaction  between 
the  conceptual  and  the  linguistic  levels  of  functioning  natural  language 
phenomena  will  be  exceptionally  difficult  to  deal  with. 

Z.fL,  Conclusion 

ffe  have  tried  to  propose  one  possible  way  to  deal  with  time  in  natural 
language,  an  approach  that  has  been  based  upon  linguistic-cognitive  universal 
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concepts  gleaned  from  natural  language  data.  We  feel  that  these  concepts  can 
be  put  to  use,  whether  one's  objective  is  to  represent  the  meaning  of  tine  in 
natural  language,  or  to  analyze  the  meaning. 

As  developed  at  this  point,  the  concept  of  event,  or  the  temporal  aspects 
of  the  concept  of  event,  includes  the  kernel  and  shell  components.  We  have 
attempted  to  characterize  how  aspectual  and  tense  meaning  of  time,  or  the 
elements  of  the  kernel  component,  fit  into  the  natural  language  description  of 
event.  We  feel  that  the  treatment  given  the  kernel  component  is  probably 
already  adequate,  with  the  exception  of  the  modal  categories,  which  is 
receiving  continued  attention.  As  regards  the  shell  component,  we  may  have 
just  touched  the  surface;  while  our  treatment  of  the  auxiliary  system  of 
modifiers  of  the  core  meaning  of  event  seems  both  systematic  and  intuitively 
appealing  (see  Diagram  IV),  we  cannot  say  at  this  point  whether  or  not  we  have 
found  all  the  auxiliary  functions  possible.  Our  treatment  of  the  adverbial 
modifiers  is  also  highly  tentative. 

In  reports  to  come  we  will  be  able  to  say  more  about  these  areas.  Our 
ongoing  efforts  emphasize  implementation  of  the  ideas  reported  in  the  paper. 
The  idea  of  approaching  the  natural  language  meaning  of  time  by  using 
conceptual  universals  is  an  extremely  appealing  idea,  and  we  except  to  be  able 
to  report  on  the  outcome  of  implementation  attempts  in  the  near  future. 


ON'K  Progress  Report 


-35- 


L.  Maran,  D.  Spoor  and  D.  Waltz 


Diagram  I:  A  Conceptual  Model  of  Time  in  ML 

A.  Word-sense  Meaning  of  Time 


TIME 


durativo  punctual 


B.  Gra-.mt  leal -Aspectual  Meaning  of  Tire 
TIME 


C.  G  r  a  mm.a  1 1  c  a  1  -  T  ■  ni  s  e  Moanin'--  of  T;'r;c 


TIME 


(present)  (past)  (future 

possible 

habitual) 


Explanation.  Since  the.  grammatical  process  is  not  a  werd-f jr  ing  process, 
grammatical  mechanisms  are  realized  in  NL  as  function  words  (e . .  the  prepositions 
of  time  --since,  until,  at,  from;  conjunctivas  such  as  --then,  before,  after--) 
and  auxiliary  mechanisms  which  use  verbs  to  modify  the  meaning  of  the  main  verb 
(MV).  Thus,  starting  tc  eat,  finishing  doing  the  laundry,  arc  expressions  utilizin 
auxiliaries  (start ,  finish)  for  the  purpose  of  encoding  the  aspectual  information 
of  boundary  reference  and  progressivity . 


Explanation:  The  temporal,  meaning  underlying  each  verbal  construction,  underlined  in  the  examples,  is 
represented  in  each  case  by  means  of  specifications  of  aspect-tense  values  of  time. 

This  illustration  reflects  a  very  preliminary  stage  of  development  of  our  ideas  as  regards 
the  conceptual  representation  of  time  in  NL.  The  potential,  as  well  as  the  limitations, 
of  the  approach  are  being  investigated. 
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v[  'suppose'  [  'plan'  [  'try'  '[KERNEL]  Adverbial  Modifier  System  ]  ]  j  •? 
AUX- z  AUX .  -y  AUX . 

g  real-time _ 

§  modal  time _ § 
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8.  Natural  Language  Front-End  for  a  High-Level  Robot  Language 

8.1.  Introduction 

To  date,  most  robot  manipulators  are  programmed  in  a  rather  primitive 
fashion.  Usually  the  robots  are  led  through  a  sequence  of  steps  by  a  human  and 
the  machine  remembers  the  exact  position  of  the  motion  and  plays  back  this 
sequence  when  called  upon  to  perform  the  task.  Of  course,  with  such  a  system, 
there  is  no  need  for  feedback  from  the  environment  as  long  as  the  situations 
the  robot  encounters  are  identical.  In  the  design  of  robust  robotic  systems, 
it  is  highly  desirable «to  include  a  comprehensive  feedback  network,  vision 
(Maddox,  1982]  or  tactile,  to  aid  the  automation  in  making  decisions  and 
allowing  for  real-time  changes  to  any  planned  motion.  In  the  not  too  distant 
future  Artificial  Intelligence  research  will  contribute  towards  the 
development  of  intelligent  autonomous  robot  systems.  Current  research  at  CSL 
will  investigate  a  high-level  natural  language  interface  with  such  a  robot 
system. 

8 .2 .  Goals 


We  believe  that  it  is  possible  and  realistic  with  the  state-of-the-art  in 
Artificial  Intelligence  to  design  a  natural  language  system  to  allow  a  human 

to  speak^  to  a  robot  [Winograd,  1970],  to  express  methods  and  goals  for  a 
complex  task,  and  to  allow  the  system  to  query  the  human  and  use  feedback  to 
successfully  complete  the  task.  The  effectiveness  of  such  a  system  will  depend 
heavily  on  how  well  humans  can  communicate  with  the  robot  via  the  natural 
language  system.  Most  tasks  are  far  too  complex  for  humans  to  identify  details 
below  some  intermediate  level.  It  has  been  shown  that  a  high-level  robot 
programming  language  can  significantly  reduce  the  detail  of  interaction 
between  some  supervisory  system  (human,  or  otherwise)  and  the  robot  worker 
[Finkel,  1974;  Geschke,  1979;  Liberman,  1976;  Paul,  1976].  Taking  things  one 
step  further,  we  envision  that  a  natural  language  system  having  some  explicit 
representation  for  goals  and  methods  could  aid  in  developing  the  algorithm  for 
accomplishing  tasks.  We  envision  an  interactive  system,  so  that  when  the 
system  finds  that  it  cannot  complete  any  sub-goal  or  the  overall  goal,  it  will 
request  additional  information  from  a  vision  system,  touch  sensing  system,  or 
through  questioning  its  instructor.  There  is  also  a  possibility  of  using 
visual  and  tactile  sensing  to  watch  an  instructor  guiding  it  or  showing  the 
robot  how  to  solve  the  task.  This  could  either  involve  watching  a  human  solve 
the  task,  or  having  the  human  guide  the  robot,  or  both. 

£.2,.  Methods 


These  investigations  will  be  carried  out  in  the  following  manner: 

(1)  Determine  a  domain  of  tasks  which  will  demonstrate  the  concept. 

(2)  Develop  the  necessary  protocol  for  the  instructor/student  relationship. 
Some  of  this  may  be  borrowed  from  human  to  human  learning,  but  new  human 

**For  now,  only  typed  input  will  be  used,  however. 
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to  machine  and  machine  to  human  concepts  will  be  included. 


(3)  Define  a  data  base  consistent  with  (1)  and  (2)  above  that  is  sufficient 
to  allow  the  machine  to  interact  with  the  environment  it  is  to  learn 
from.  The  knowledge  base  for  the  natural  language  front  end  will  have  to 
be  as  complete  as  possible  for  its  task  domain,  and  the  range  of  commands 
of  the  robot  language  should  be  large.  This  includes  determining  what 
non-English  feedback  is  necessary  and  what  form  should  it  maintain 
(vision  and  tactile  information,  data  compression,  etc.). 

(4)  Develop  and  test  the  proposed  system. 

8.4.  Accomplishments 

Presently  we  are  identifying  several  tasks  which  will  demonstrate  that  a 
robot  has  learned  sufficiently  about  a  goal  to  accomplish  a  task.  In  the 
beginning,  we  expect  to  use  the  protocol  from  several  exchanges  with  people  to 
gain  a  feel  for  how  information  is  transferred  in  this  context.  We  expect  that 
by  January  1 9 T3 ,  we  will  know  the  tasks  and  have  a  protocol  outlined  for  each. 
It  is  anticipated  that  we  will  use  some  high-level  robot  language  which  has 
previously  been  developed,  and  will  add  to  it  if  necessary  to  enhance  its 
capability.  We  will  use  a  Stanford  robot  arm  is  conjunction  with  visual 
information  from  one  or  two  solid-state  cameras.  Though  color  processing  would 
be  desirable,  most  likely  we  will  use  only  binary  or  gray-level  visual  data 
for  now. 

8.4.1.  Future  Work 

We  anticipate  that  our  research  will  provide  a  foundation  for  integrating 
many  robotic  sub-systems  and  demonstrate  that  such  a  composite  system  can  be 
user-friendly  to  humans  and  require  a  minimum  of  interaction  and  detail.  We 
hope  to  identify  major  portions  of  this  natural  language  system,  and  give 
detailed  specifications  of  those  components  so  that  they  may  be  used  in  other 
systems . 
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